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Abstract 

Given an infinitesimal perturbation of a discrete-time finite Markov 
chain, we seek the states that are stable despite the perturba¬ 
tion, i.e. the states whose weights in the stationary distributions 
can be bounded away from 0 as the noise fades away. Chemists, 
economists, and computer scientists have been studying irreducible 
perturbations built with exponential maps. Under these assump¬ 
tions, Young proved the existence of and computed the stable states 
in cubic time. We fully drop these assumptions, generalize Young’s 
technique, and show that stability is decidable as long as / £ 0{g) 
is. Furthermore, if the perturbation maps (and their multiplications) 
satisfy / G 0{g) or g £ 0{f), we prove the existence of and com¬ 
pute the stable states and the metastable dynamics at all time scales 
where some states vanish. Conversely, if the big-O assumption 
does not hold, we build a perturbation with these maps and no sta¬ 
ble state. Our algorithm also runs in cubic time despite the general 
assumptions and the additional work. Proving the correctness of 
the algorithm relies on new or rephrased results in Markov chain 
theory, and on algebraic abstractions thereof. 

Categories and Subject Descriptors G [d] 

General Terms 

Keywords evolution, learning, metastability, tropical algebra, 
shortest path, SCC, cubic time algorithm 

1. Introduction 

Motivated by the dynamics of chemical reactions, Eyring d and 
Kramers flal studied how infinitesimal perturbations of a Markov 
chain affects its stationary distributions. This topic has been further 
investigated by several academic communities including probabil¬ 
ity theorists, economists, and computer scientists. In several fields 
of application, such as learning and game theory, it is sometimes 
unnecessary to describe the exact values of the limit stationary dis¬ 
tributions: it suffices to know whether these values are zero or not. 
Thus, the stochastically stable states (d, m, Cl) were defined 
in different contexts as the states that have positive probability in 
the limit. We rephrase a definition below. 


* This author is supported by the ERC inVEST (279499) project and was in 
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Definition 1 (Markov chain perturbation and stochastic stability) 

Let I be a subset of positive real numbers with 0 as a limit point for 
the usual topologjij^ A perturbation is a family {{Xn^)n£ti)eei of 
discrete-time Markov chains sharing the same finite state space. If 
the chain is irreducible for all e £ I, then {{Xn'')neti)eei 

is said to be an irreducible perturbation. 

A state X of {{Xn'')nev)eei is stochastically stable if there 
exists a family of corresponding stationary distributions 
such that liminf£_>o Pe{x) > 0. It is stochastically fully vanishing 
//■ lim sup,j_>Q/ie(x) = 0 for all {fj,e)eei- Non-stable states are 
called vanishing. 

Definition[T]may be motivated in at least two ways. First, a dynam¬ 
ical system {e.g. modeled by a Markov chain) has been perturbed 
from the outside, and the laws governing the systems {e.g the tran¬ 
sition probability matrix) have been changed. As time elapses {i.e. 
as e approaches zero), the laws slowly go back to normal. What 
are the almost sure states of the system after infinite time? Second, 
a very complex Markov chain is the sum of a simple chain and a 
complex perturbation matrix that is described via a small, fixed ep • 
The stationary distributions of the complex chain are hard to com¬ 
pute, but which states have significantly positive probability after 
infinite time? Our main result below answers these questions. 

Theorem 2 Consider a perturbation such that f £ 0{g) or g £ 
0{f) for all f and g in the multiplicative closure of the transition 
probability functions e i—>■ Pe{x, y) with x y. Then the perturba¬ 
tion has stable states, and stability can be decided in 0(n®), where 
n is the number of states. 

Note that by finiteness of the state space it is easy to prove that 
every perturbation has a state that is not fully vanishing. 

1.1 Related works and comparisons 

In 1990 Foster and Young a defined the stochastically stable states 
of a general (continuous) evolutionary process, as an alternative to 
the evolutionary stable strategies flm . Stochastically stable states 
were soon adapted by Kandori, Mailath, and Rob llOr for evolu¬ 
tionary game theory with 2x2 games. Then Young d. Theo¬ 
rem 4] proved ”a finite version of results obtained by Freidlin and 
Wentzel” in @]. Namely, he characterized the stochastically sta¬ 
ble states if the perturbation satisfies the following assumptions: 1) 
the perturbed matrices P'^ are aperiodic and irreducible; 2) the P’^ 
converge to the unperturbed matrix when e approaches zero; 3) 
every transition probability is a function of e that is equivalent to 
c- e“ for some non-negative real numbers c and a. The main tool in 
Young’s proof was proved by Kohler and Vollmerhaus m and is 
the special case for irreducible chains of the Markov chain tree the¬ 
orem (see fisil or Si). Young’s characterization involves minimum 
directed spanning trees, which can be computed in 0{n^) 0] for 


^ This implies that I is infinite. ]0,1] and {^ | n- G N} are typical I. 
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graphs with n vertices. Since there are at most n roots for directed 
spanning trees in a graph with n vertices, Young can compute the 
stable states in 0{n^\ 

In 2000, Ellison jj] characterized the stable states via the al¬ 
ternative notion of the radius of a basin of attraction. The major 
drawback of his characterization compared to Young’s is that it is 
’’not universally applicable” d ; the advantages are that it provides 
”a bound on the convergence rate as well as a long-run limit” and 
’’intuition for why the long-run stochastically stable set of a model 
is what it is”. In 2005, Wicks and Greenwald d designed an al¬ 
gorithm to express the exact values of the limit stationary distri¬ 
bution of a perturbation, which, as a byproduct, also computes the 
set of the stable states. Like d they consider perturbations that 
are related to the functions e i—t e“, but they only require that the 
functions converge exponentially fast. Also, instead of requiring 
that the P' be irreducible for e > 0, they only require that they 
have exactly one essential class. They do not analyze the complex¬ 
ity of their alg orithm but it might be polynomial time. We improve 
upon [id, ( 1 , and d in several ways. 

1. The perturbation maps in the literature relate to the maps e i—t 

. Their specific form and their continuity, especially at 0, are 
used in the existing proofs. Theoremj^dramatically relaxes this 
assumption. Continuity, even at 0, is irrelevant, which allows 
for aggressive, i.e., non-continuous perturbations. We show that 
our assumption is (almost) unavoidable. 

2. The perturbations in the literature are irreducible (but d 
slightly weakened this assumption). It is general enough for 
perturbations relating to the maps e i—t e“, since it suffices to 
process each sink (aka bottom) irreducible component inde¬ 
pendently, and gather the results. Although this trick does not 
work for general perturbation maps, Theoreml^manages not to 
assume irreducibility. 

3. The perturbation is abstracted into a weighted graph and shrunk 
by combining recursively a shortest-path algorithm (w.r.t. some 
tropical-like semiring) and a strongly-connected-component al¬ 
gorithm. Using tropical-like algebra to abstract over Markov 
chains has already been done before, but not to solve the sta¬ 
ble state problem, (d did it to prove an algebraic version of 
the Markov chain tree theorem.) 

4. Our algorithm computes the stable states in 0(n®), as in 
which is the best known complexity. In addition, the compu¬ 
tation itself is a summary of the asymptotic behavior of the 
perturbation: it says at which time scales the vanishing states 
vanish, and the intermediate graph obtained at each recursive 
stage of the algorithm accounts for the metastable dynamics of 
the perturbation at this vanishing time scale. 

Section 11.21 sets some notations; Section 11.31 analyses which as¬ 
sumptions are relevant for the existence of stable states; Section[3 
proves the existential part of Theorem|2l i.e. it develops the proba¬ 
bilistic machinery to prove the existence of stable states; hinging on 
this, Section[^proves the algorithmic part of Theorem|2l i.e. it ab¬ 
stracts the relevant objects using a new algebraic structure, presents 
the algorithm, and proves its correctness and complexity; Section|4] 
discusses two important special cases and an induction proof prin¬ 
ciple related to the termination of our algorithm. 

1.2 Notations 

• The set N of the natural numbers contains 0. For a set S and 
n £ N, let S'™ be the words 7 over S of length I 7 I = n. Let 
S* ~ UngNS" be the finite words over S. The set-theoretical 
notation UE := Uxsex is used in some occasions. 

• Let be a Markov chain with state space S. For all 

A C S let TA ~ inf{n > 0 : X„ G A} (r^ := inf{n > 0 : 


Xn £ A}) be the first time (first positive time) that the chain 
hits a state inside A. Usually and are written Tx and 
Tx, respectively. 

• Given a Markov chain (-Y„)neN. the corresponding matrix rep¬ 

resentation, law of the chain when started at state x, expectation 
when started at state x, and possible stationary distributions are 
respectively denoted p, E'^, and p. When considering other 
Markov chains or (.Y„)„gM, the derived notions are 

denoted with tilde or circumflex, as in p or p. 

• A perturbation ((ATi'^j^g^jeg/ will often be denoted X for 
short, and when it is clear from the context that we refer to a 
perturbation, p will denote the function {e,x,y) 1 —)■ Pe{x,y) 
(instead of {x, y) 1 —t p{x, y)), and p{x, y) will denote e i-t- 
Pe{x,y) (instead of a mere real number). The other derived 
notions are treated likewise. 

• The probability of a path is defined inductively by p{xy) := 
p(x,y) and p{xy^) := p{x,y)p{yj) for all x,y £ S and 
7 G 5 X 5 *. 

• Given x, y, and a set A, a simple A-path from 2 : to t/ is a 
repetition-free (unless x = y) word 7 starting with x and 
ending with y, and using beside x and y only elements in A. 
Formally, VA{x,y) ~ {7 G {a;} x A* x {y} | (1 < i < J < 
I7I A 7i = 7/) ^{i = l^j = I7I)}. 


1.3 Towards general assumptions 

A state 2 : of a perturbation is stable if there exists a related family 
of stationary distributions such that p{x) = 0 ( 1 ), but 
even continuous perturbations that converge when e approaches 0 
may fail to have stable states. For instance let S := {x, y} and 
for all e g]0, 1 ] let p^jx, y) := and pe(t/, 2;) := g^+costE ) 
as in Figure [T3 where the self-loops are omitted. In the unique 
stationary distribution 2; has a weight ^£(2;) = (l-t-e”™®*-' 

Since 1 ( 2 ;) = 1 and( 2 ;) = 

i+ 2 (n+i)Tr ^"^00 0 , neither x nor y is stable. 

As mentioned above, the perturbations in the literature are re¬ 
lated to the functions e 1 —t e“ with a > 0 , which rules out 
the example from Figure [T^ and implies the existence of a stable 
state El. Here, however, we want to assume as little as possible 
about the perturbations, while still guaranteeing the existence of 
stable states. Towards it let us first rephrase the big O notation as a 
binary relation. It is well-known that big O enjoys various algebraic 
properties. The ones we need are mentioned in the appendix. 

Definition 3 (Order) For /, g : 7 —>■ [0,1], let us write f ^ g if 
there exist positive b and e such that f{e) < b-g{e)for all e < e; 
let f = g stand for f ^ g F g ^ f. 


Requiring that every two transition probability maps / and g occur¬ 
ring in the perturbation satisfy / A g or g A / rules out the exam¬ 


ple from Figure [T^ but not the one from Figure [Tbl There p^ (z) < 


p,{x) = 
ge{z) 


and p^{y) = 


-. So 


1 + e“s(' U(l + e2) r- w/ i_g,;Cos(e 1)(1 + £2)- 

.^0 0 and p2n7r(.y'} i'7i—¥oo 0 and p 2 ('n,.Vt)'!T(.x') t'Ti—>oo 

0, no state is stable. Informally, « is not stable because it gives ev¬ 
erything but receives at most e; neither x nor y is stable since their 
interaction resembles Figure [T^due to e® and e"* • g 2 -l-cos(e ) 
remark is turned into a general Observation |4]below. 


Observation 4 For 1 < i < n and 1 < j < m let fi, gj : I —>■ 
[0, 1] be such that fi and gj are not ^-comparable. Then 
there exists a perturbation without stable states that is built only 
with the fi,..., fn,gi,... ,gm and the I — fi,. .. ,1 — fn,l - 
pi, . . . , 1 — gm- See Fisure\Td\ 
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(d) 

Figure 1: Perturbations without stable states 




Figure 2: Perturbations with stable states 

Observation|4]motivates the following ’’unavoidable” assumption. 

Assumption 5 The multiplicative closure of the maps e Pe{x,y) 

with X y is totally preordered by 

For example, the classical maps e i—^ c • e“ with c > 0 and 
q: £ R constitute a multiplicative group totally preordered by 
One reason why we can afford such a weak Assumption |5] is that 
we are not interested in the exact weights of some putative limit 
stationary distribution, but only whether the weights are bounded 
away from zero. 

Let us show the significance of Assumption |5] which is satis¬ 
fied by the perturbations in Figure [^andlSH Young’s result shows 
that y is the unique stable state of the perturbation in Figure |2^ but 
it cannot say anything about Figures and[^ Figure [2bl is 

not regular, i.e., does not converge, and neither do the 

weights ptix) and j-Le{y), but it is possible to show that both limits 
inferior are 1 /4 nonetheless, so both x and y are stable; the tran¬ 
sition probabilities in Figure]^ do not converge, and —1 

and 1 — —I are not even comparable, but it is easy to see 

that p.e{x) = pe(t/) = ^; and in Figure!^ a: is the only stable state 


since its weight oscillates between i and 1. Note that Assumption!^ 
rules out the perturbations in Figure[T] which have no stable state. 

2. Existence of stable states 

This section presents three transformations that simplify pertur¬ 
bations while retaining the relevant information about the stable 
states. Two of them are defined via the dynamics of the original 
perturbation. The relevance of these two transformations relies on 
the close relation between the stationary distributions and the dy¬ 
namics of Markov chains. Lemmaj^below pinpoints this relation. 

Lemma 6 A distribution p of a finite Markov chain is stationary 
iff its support involves only essential states and for all states x and 
y we have /r( 2 ;)P''(r+ < t+) = At(j/)P^(r+ < t+). 

Lemmaj^can already help us find the stable states of small exam¬ 
ples such as in Figures[T]and[2] In Figure[T^it says that pe{x)e^ — 
so we find lim inf/ie(a;) = liminf/ie(y) = 0 
without calculating the stationary distributions. In Figure|^it says 
that pe{x){2 — cos(e“^)) = /re(t/)(2 -|- cos(e“^)), so pe{x) < 
3fj,e{y) and_j < p,e{y), and likewise for x. 

Lemma|3below shows further connections between the station¬ 
ary distributions and the dynamics of Markov chains. Its proof in¬ 
volves Lemma| 6 ] and its irreducible case is used in Section l23] 

Lemma 7 Let p be a Markov chain with state space S, and let p 

be defined over S Q S by p{x, y) := P“’(A' + = y). 

’’s 

1. Then P^ (jy < rf) ='P^ (xy < rf) for all x,y G S. 

2. Let pip) be a stationary distribution for p (p). IfS are essential 
states, there exists p (p) a stationary distribution for p (p) such 
that p{x) = p{x) ■ '^y^s Tiy)fa’' X G S. 

The dynamics, i.e., terms like P'^(r+ < r+) or P^(2f^+ = y) 
are usually hard to compute, and so will be the two transformations 
that are defined via the dynamics, but Lemma[ 8 ]below shows that 
approximating them is safe as far as the stable states are concerned. 

Lemma 8 Let p and p be perturbations with the same state space, 
such that X 7 ^ y => p{x, y) = p{x, y). For all stationary distribu¬ 
tion maps pforp, there exists pforp such that p = p. 

E.g., both coefficients in Figure l^dbbb can safely be replaced with 
e (1), and Figure|40can be replaced with Figure |4H Lemma[8]will 
dramatically simplify the computation of the stable states. 

2.1 Essential graph 

The essential graph of a perturbation captures the non-infinitesimal 
flow between different states at the normal time scale. It is a very 
coarse description of the perturbation. 

Definition 9 (Essential graph) Given a perturbation with state 
space S, the essential graph is a binary relation over S and pos¬ 
sesses the arc (x, y) ifx y andp{z, i) F p(^x, y) for all z,t G S. 
The essential classes are the sink (aka bottom) strongly connected 
components of the graph. The other SCCs are the transient classes. 
A state in an essential class is essential, the others are transient. 

The essential classes will be named E\,..., Ek. Observation [Tol 
below implies that the essential graph is made of the arcs (x, y) 
such that X jk y and p{x, y) = 1, as expected. 

Observation 10 Letp be a perturbation. There exist positive c and 
£o such that for all e < co.for all simple paths 7 in the essential 
graph, c < Pe( 7 ). 
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For example, the perturbations (with I =]0,1]) that are described in 
Figures[T 2 [T 2 |^ andl^all have Figure!^ as essential graph, and 
{x} and {y} as essential class. Figure [^ ( 1^ is the essential graph 
of Figure |4a](|^, and {x, y} and {f} are its essential classes. Note 
that the essential states of a perturbation and the essential states of a 
Markov chain are two distinct (yet related) concepts: e.g., all states 
from Figure|4a|are essential for the Markov chain for all e £]0,1]. 

The essential graph alone cannot tell which states are stable: 
e.g., swapping e and in Figure|^yields the same essential graph 
but Lemma shows that the only stable state is then x instead 
of y. The graph allows us to make the following case disjunction 
nonetheless, along which we will either say that all states are stable, 
or perform one of the transformations from the next subsections. 

1. Either the graph is empty (i.e. totally disconnected) and the 
perturbation is zero, or 

2 . it is empty and the perturbation is non-zero, or 

3. it is non-empty and has a non-singleton essential class, or 

4. it is non-empty and has only singleton essential classes. 

Observation [To]motivates the following convenient assumption. 

Assumption 11 There exists c > 0 such that p{'y) > cfor every 
simple path 7 in the essential graph. 

The two assumptions above do not have the same status: Assump¬ 
tion!^ is a ksy condition that will appear explicitly in our final re¬ 
sult, whereas Assumption[TT]is just made wlog, i.e., up to focusing 
on a smaller neighborhood of 0 inside I. 

LemmafT^ shows the usefulness of As sumption [TT] It is proved 
by Lemmaj^ and is used later to strengthen Lemma|7||2|into p = ft. 

Lemma 12 Let a perturbation p with state space S and transient 
states T satisfy Assumution U 1\ Then < '^xes\T l^i^)- 



Figure 4: Essential collapse 


Eor example, collapsing around x or y in Eigure|2b|has no effect. 
The perturbation in Figure|4^has two essential classes, i.e., its es¬ 
sential graph has two sink SCCs, namely {x, y} and {f}. Figure 1^ 
displays its essential collapse around x. It was calculated by notic¬ 
ing that = t) = ^, and P^{X^+ = x) = 


+ =x),andP«(X+ = x) = 

{x,z,t} {x,z,t} 


i + i^.pv(x. =x). 

t x.z.tt 

Proposition [18] will show that it suffices to compute the stable 
states of Figure|4b|to compute those of Figure|4a| and by Lemmaj^ 
it suffices to compute those of the simpler Figure [4^ However, 
computing the exact values P^(X + = u) can be difficult 

^S\EU{x} 

even on simple examples like above. Fortunately, Lemmal 15 [shows 
that they are ^-equivalent to maxima that are easy to compute. 
E.g., using Lemma to approximate the essential collapse of 
Figure!^ around x yields Figure|4^ but without having to compute 
the intermediate Figure!^ 


Lemma 15 Let p be a perturbation with state space S satisfy 
Assumption |5| and let p be the essential collapse rtijp, x) of p 
around x in some essential class E. For all y £ S \ E, we have 
p{UE,y) = maxzeEp{z,y) andp{y,UE) = maxxf=EPiy,z). 

Note that by Lemma [Tsl only the essential class is relevant dur¬ 
ing the essential collapse up to =, the exact state is irrelevant. 
Lemma[T^is also a tool that is used to prove, e.g.. Proposition [7^ 
below which shows that the essential graph may contain useful in¬ 
formation about the stable states. 


2.2 Essential collapse 

The essential collapse, defined below, amounts to merging one 
essential class of a perturbation into one meta-state and letting 
this state represent faithfully the whole class in terms of dynamics 
between the whole class and each of the outside states. 

Definition 13 (Essential collapse of a perturbation) Let p be a 

perturbation on state space S. Let x be a state in an essential 
class E, and let S ~ {S \ E) U {UE}. The essential collapse 
k(p, x) : 1 X .5 X S —>■ [0,1] ofp around x is defined below. 
n{p,x){uE,UE) :=P^(X + = x) 

’'s\EU{x} 

h:(p,x)(UE,i/) := P“'(X + = y) for ally £S\E 

^S\EU{x] 

«(P, x) {y, UE) :='^p{y,z) forally £ S\E 

zGE 

k(p, x){y, z) := p{y, z) for ally,z £ S\E 

Observation 14 k(p, x) is again a perturbation, k preserves irre- 
ducibility, and if {x} is an essential class, nijp, x) = p. 


Proposition 16 Let a perturbation p with state space S satisfy 
Assumption\^ let pbe a corresponding stationary distribution map. 

1. Ify is a transient state, liminfe_»o pt(y) = 0 . 

2. If two states x and y belong to the same essential or transient 
class, p{x) = p{y). 

Proposition ! 1611 I says that the transient states are vanishing, e.g. the 
nameless states in Figure [3cl Proposition 11612! says that two states 
in the same class are either both stable or both vanishing, e.g. {x} 
and { 1 /} in Figure [3bl 

The usefulness of the essential collapse comes from its preserv¬ 
ing and reflecting stability, as stated in Proposition 118! Its proof 
invokes Lemma [FTI below, which shows that the essential collapse 
preserves the dynamics up to =, and Lemma which relates the 
dynamics and the stationary distributions. 

Lemma 17 Given a perturbation p with state space S, let p be the 
essential collapse ofp around x in some essential class E, and let 
X := \JE. The following holds for all y £ S \ E. 

P“(tx < Ty) ~ P“(ts < Ty) A P'^jry < Tx) = P'^('rj, < n) 
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Proposition 18 Let a perturbation p with state space S satisfy 
Assumption]^ and let x be in an essential class E. 

1. Let p be the chain after the essential collapse of p around 
X. Let p (ft) be a stationary distribution map of p (p). There 
exists a stationary distribution map fi for p {p for p) such that 
p{UE) = p{x) and p{y) = p{y) for all y £ S \ E. 

2. A state y £ S is stable for p iff either y £ E and UE is stable 
for k{p, x), ory^E and y is stable for k{p, x). 

By definition, collapsing an essential class preserves the structure 
of the perturbation outside of the class, so Proposition 1181 implies 
that the essential collapse commutes up to Especially, the order 
in which the essential collapses are performed is irrelevant as far as 
the stable states are concerned. 

2.3 Transient deletion 

If all the essential classes of a perturbation are singletons, Observa- 
tion[T4]says that the essential collapse is useless. If in addition the 
essential graph has arcs, there are transient states, and Definition! 191 
below deletes them to shrink the perturbation further. 

Definition 19 (Transient deletion) Let a perturbation p with 
state space S, transient states T, and singleton essential classes, 
satisfy Assumption]^ The function S{p) over S\T is derived from 
p by transient deletion: for all distinct x,y £ S \ T let 

S{p){x,y) + =y) 

S\T 

Observation 20 5{p) is again a perturbation, S preserves irre- 
ducibility, and if all states are essential, S{p) — p. 

For example, in Figure the essential classes are {x} and {y}, 
z is transient, and the transient deletion yields Figure]^ Also, in 
Figure [5^ the essential classes are {a;}, {y}, and {z}, the transient 
states are nameless, and the transient deletion yields Figure [5^ 

The transient deletion is useful thanks to Propositionl21lbelow. 
whose proof relies on Lemmas l7l2l and[T^ 

Proposition 21 If a perturbation p satisfy Assumption and has 
singleton essential classes, p and S{p) have the same stable states. 

Like the essential collapse, the transient deletion is defined via 
the dynamics and is hard to compute. Like Lemma [TS] did for the 


essential collapse. Lemma ]T2\ approximates the transient deletion 
by an expression that is easy to compute. 

Lemma 22 If a perturbation p with state space S and transient 
states T satisfies Assumption]^and has singleton essential classes, 

+ = y) ^ max{p( 7 ) : 7 G FrCa;,?/)} for all x,y £ S\T. 

S\T 

£.g., Figure[^yields Figure|5c]without computing Figure[50 Note 
that max(e^ > f) in Figurel5c]mav be simplified into e by Lemma[8] 

2.4 Outgoing scaling and existence of stable states 

If the essential graph has no arc, the essential collapse and the 
transient deletion are useless to compute the stable states. This 
section says how to transform a non-zero perturbation with empty 
(i.e. totally disconnected) essential graph into a perturbation with 
the same stable states but a non-empty essential graph, so that 
collapse or deletion may be applied. Roughly speaking, it is done 
by speeding up time until a first non-infinitesimal flow is observable 
between different states, i.e. until the new essential graph has arcs. 

Towards it, the ordered division is defined in Definition It 
allows us to divide a function by a function with zeros by returning 
a default value in the zero case. It is named ordered because we will 
’’divide” f hy g only if f g, so that only 0 may be ’’divided” by 
0. Then Observation |24]further justifies the terminology. 

Definition 23 (Ordered division) For /, gr : I —>■ [0, 1] and n > 

1 let us define {f g) : I ^ [0, 1] by (/ g){x) := ^ if 

0 < g{x) and otherwise (/ -£n g){x) := 

Observation 24 (/ g) ■ g — f for all n and /, g : 7 —>■ [0,1] 
such that f if: g. 

Definition 25 (Outgoing scaling) Let a perturbation p with state 
space S satisfy Assumption]^ let m := IS] • max{p(z, t) \ z,t £ 

S A z t}, and let us define the following. 

• o-{p){x,y) ~p{x,y) ^|s| mforallxyl y 

• o-{p)ix,x) := {pix,x) +m-l) -G|s| m. 

For example. Figure satisfies Assumption |5] and its essential 
graph is empty, i.e. totally disconnected. Applying outgoing scaling 
to it yields Figure which satisfies Assumption |5] and whose 
essential graph has two arcs. Note that collapsing around x oi y 
in Figure |2^ has no effect, but in Figure it yields a one-state 
perturbation. Also, Figure 1^ does not satisfy Assumption [5] and 
its essential graph is empty. Applying outgoing scaling to it yields 
Figure[6^ which does not satisfy Assumption|5]and whose essential 
graph has one arc. Applying it again to Figure|6^ would only divide 
the non-self-loop coefficients by 3. More generally. Proposition!^ 
below states how well the outgoing scaling behaves. 

Proposition 26 I. If a perturbation p satisfies Assumption]^ so 
does <j{jp), and the essential graph of<j{jp) is non-empty . 

2. A state is stable for p iff it is stable for cr{p). 

The outgoing scaling divides the weights of the proper arcs by m, 
as if time were sped up by m~^. The self-loops thus lose their 
meaning, but Propositionl26|proves it harmless. Note that the self¬ 
loops are also ignored in Assumptionj^ Lemma[^ and Definition!^ 
Let us now describe a recursive procedure computing the stable 
states: if the perturbation is zero, all its states are stable; else, if the 
essential graph is empty, apply the outgoing scaling; else, apply one 
essential collapse or the transient deletion. This procedure is correct 
by ProDositions l26l2l| 1 8121 and|2T] hence Theoreml27lbelow. which 
is the existential part of Theorem|2| 
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Figure 6: Outgoing scaling 


Theorem 27 Let p be a perturbation such that f ^ g or g ^ f for 
all / and g in the multiplicative closure of the p{x, y) with x y. 
Then p has stable states. 

3. Abstract and quick algorithm 

The procedure described before Theorem 1271 computes the stable 
states, but a very rough analysis of its algorithmic complexity 
shows that it runs in 0{rJ), where n is the number of states. 
(A better analysis might find Oirf).) This bad complexity comes 
from the difficulty to analyze the procedure precisely and from 
some redundant operations done by the transformations, especially 
the successive essential collapses. Instead we will perform the 
successive collapses followed by one transient deletion as a single 
transformation. Applying alternately the outgoing scaling and the 
new transformation, both up to =, is the base of our algorithm. 

Section 13.11 abstracts the relevant notions up to = and gives 
useful algebraic properties that they satisfy. Based on these ab¬ 
stractions, Section[T2]presents the algorithm (computing the stable 
states and more), its correctness, and its complexity in Ofnf). 

3.1 Abstractions 

Ensuring that the essential collapse and the transient deletion can 
be safely performed up to = is a straightforward sanity check, by 
Lemma [8] However, the proof for the outgoing scaling involves a 
new algebraic structure to accommodate the ordered division, and 
handling the combination of the successive collapses and one dele¬ 
tion requires particular attention. It would have been cumbersome 
to define this combination directly via the dynamics in Section 
and more difficult to prove its correctness via probilistic techniques, 
hence the usefulness of the rather atomic collapse and deletion. 

Our firsts step below is to consider functions up to =. 

Definition 28 (Equivalence classes and quotient set) For f 

I —>■ [0,1] let [/] be its = equivalence class; for a matrix 
A = with elements in I [0,1], let [A] be the 

matrix where [A]ij := [aij] for all 0 < i,j < n. For a set F of 
functions from I to [0, 1], let [F’] be the quotient set F/ =. Finally, 
it is possible to lift over [F] both ■ to [■] and A to [A]. 

Observation 29 For (G, •) a semigroup totally preordered by 

1. [A] orders [G] linearly, so max[^] is well-defined. 

2. ([G] U {[e i-> 0]}, [e 1 -^ [e !->■ 1], max[^], [•]) is a commuta¬ 
tive semiring. (See, e.g., ^ for the related definitions.) 


The good behavior of • and A up to = is expressed above within 
an existing algebraic framework, but for we introduce a new 
algebraic structure below. 

Definition 30 (Ordered-division semiring) An ordered-division 
semiring is a tuple (F’, 0,1, •, <, 3-) such that (F’, <) is a lin¬ 
ear order with maximum 1, and (F', 0,1, max<, •) is a commu¬ 
tative semiring, and for all f < g we have f ^ g is in F and 
if Fg)-g = f. 

Observation 31 Let (F, 0,1, •, -F, <) be an ordered-division semir¬ 
ing. Then 0 = min< F and / -F 1 = f for all f. 

Lemma[32]below shows that the functions I —^ [0,1] up to = form 
an ordered-division semiring. 

Lemma 32 1. Let n > 1 and f,f',g,g' ■ I —^ [0,1] be such 
that f ^ f ;< g ^ g'. Then [f -Fi g] = [/' -Fn g'], which we 
then write [/][-f][( 7 ]. 

2. For all sets G of functions from 7 to [0, 1] closed under multipli¬ 
cation, the tuple ([GU{e i-^ 0}], [e 0], [e !->■ 1], [•], [a], [;^]) 
is an ordered-division semiring. 

For example, the set containing [e i—0] and all the [e i—e“] 
for non-positive a is an ordered-division semiring, where [e i—>■ 
£“][■][£ = [£'->■ and [e i->- e“][;:i][e !->■ e^] iff P < a, 

and [e i-)- 0][A][e i->. e“], and [e i-)- e“][A][e i-)- e^] = [e 
for P < a. 

To handle a, k, and J up to = we define below transformations 
of weighted graphs with weights in an ordered-division semiring. 

Definition 33 (Abstract transformations) Let P : S x S ^ F, 

where {F, 0, 1, •, <, A) is an ordered-division semiring. 

1. Let {(z, t) € I P{z, t) = 1 f\ z ^ f} be the essential graph 
of P, and let the sink SCCs Ei,..., Ek be its essential classes. 

2. Outgoing scaling: for x ^ y let [a] (P) {x, y) := P{x, y) A M, 
where M ~ ma.x<{P(^z,t) : {z,t) € S x S A z t}, and 
[a]{P){x,x) ~ 1. 

3. Essential collapse: let [K]{P,Ei) be the matrix with states 

{UFli} U S' \ 7?i such that for all x,y £ S \ E; we set 
[K\{P,Ei){x,y) := P{x,y) and [k]{P, Ei){}jEi,y) ;= 

m.&x<{P{xi,y) : Xi e Ei) and [k]{P, Ei){x,yjEi) ;= 
max<{P(a;,a;i) : Xi G E;} and\K]{P, Ei){'jEi,yjEi) ~ 1. 

4. Shrinking: let [x](F’) be the matrix with state space {UFli,..., UEk} 
such that for all i,j [y](P)(uFli, OEj) ;= max<{P( 7 ) : 7 G 
Tt{E,,E^)}. 

In Definition!^ ths weights P{x, x) occur only in the definitions 
of the self-loops of the transformed graphs, whence Observation!^ 
below. 

Observation 34 Let {F, 0,1, ■, <, a) be an ordered-division semir¬ 
ing, let P, P' : S X S ^ F be such that P{x, y) = P'{x, y)for all 
X y. Then P and P' have the same essential graph and classes 
Ei,...,Ek; [o-]{P){x,y) = [a\{P'){x,y) for all x y; and for 

all I and i ^ j we have [x]{P){k)Ei, UEj) = [x](P')(u7?i, UEj) 
and[K]iP)iP,E,){uEi,UEj) = [k]{P'){P', Ei){uEi,UEj). 

Lemmal^below shows that the transformations from Definition!^ 
are faithful abstractions of a, k, and S. Some proofs come with 
examples, which also highlight the benefits of abstraction. 

Lemma 35 (Abstract and concrete transformations) Let a per¬ 
turbation p with state space S satisfy Assumption]^ let Ei,..., Ek 
be its essential classes, and for all i let x; G E;. 

1. p and [p] have the same essential graph. 

2. [(j]{\p])(^x,y) = [cj(^p)](^x,y) for all X y. 
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[x](b])({®}. {«/}) = lS{p)]{x,y) whenever 5{p) is well- 

defined. 

4. [K]{[p],Ei) = [K{p,Xi)]. 

5- M(b]) = M°M(b],i5i). 

6. [(5 o n{... K{K{a{p),xi),X 2 ) ■ ■ ■ ,Xk)]iLlEi,UEj) = [x] o 
[o-] (b]) , 'JEj ) for all i j. 

By the algorithm underlying Theorem 1271 and Lemma [35l6l we are 
now able to state the following. 

Proposition 36 Let a perturbation p satisfy Assumption |5] There 
exists n € N such that ([x] o [a-])'^{[p\)(x,y) = Q for all x y 
in its state space. Furthermore, the states of ([x] o [cr])"([p]) 
correspond to the stable states ofp. 

3.2 The algorithm 

Algorithm [T] mainly consists in applying recursively the function 
[x] o [ct] occurring in Proposition 1361 until an empty {i.e. totally 
disconnected) graph is produced. It does not explicitly refer to 
perturbations since this notion was abstracted on purpose. Instead, 
the algorithm manipulates digraphs with arcs labeled in an ordered- 
division semiring, in which inequality, multiplication and ordered 
division are implicitly assumed to be computable. 

One call to the recursive function HubRec corresponds to [xj° 
[cr], i.e. Lines |7] and [9] correspond to [cr], and Lines [Tol till 1181 
correspond to [x]. Before calling HubRec Lines and [3] produce 
an isomorphic copy of the input, which will be easier to han¬ 
dle when making unions and keeping track of the remaining ver¬ 
tices. Note that Line does not update the P{x,x). It would be 
useless indeed, since in Definition [33] the self-loops of the orig¬ 
inal graph occur only in the definition of the self-loops of the 
transformed graphs, and since self-loops are irrelevant by Obse- 
vation (Ml Line 1101 computes the essential graph, up to self-loops, 
and Line m computes the essential classes by a modified Tar- 
jan’s algorithm, as detailed in Algorithm |2| The computation of 
[x]iP)i^Ei,UEj) ~ max<{P(7) : 7 € rT(Pi, Pj)} is per¬ 
formed in two stages: the first stage at Line[T2|considers only paths 
of length one; the second stage at Line [T8| considers only paths of 
length greater than one, and therefore having their second vertex 
in T. This case disjunction reduces the size of the graph on which 
the shortest path algorithm from Line [T^ is run, and thus reduces 
the complexity of Hub from O(n^) to 0(n®), as will be detailed in 
Proposition]^ Note that the shortest path algorithm is called with 
laws • and max instead of + and min. Moreover, since our weights 
are at most 1 we can use or U (which assume non-negative 
weights) to implement Linell6l 

Propositionl37lbelow shows that our algorithm is fast. 

Proposition 37 The algorithm Hub terminates within 0{n^) com¬ 
putation steps, where n is the number of vertices of the input graph. 

By Propositions and we now state our main algorithmic 
result, which is the algorithmic part of Theorem|2| 

Theorem 38 Let a perturbation p satisfy Assumption]^ A state is 
stochastically stable iff it belongs to Hub(S', [p]). Provided that in¬ 
equality, multiplication, and ordered division between equivalence 
classes of perturbation maps can be computed in constant time, 
stability can be decided in 0(n®), where n is the number of states. 

One of the achievements of our algorithm is that it processes all 
weighted digraphs {i.e. abstractions of perturbations) uniformly. 
Especially, neither irreducibility nor any kind of connectedness is 
required. For example in Figure |3 the four-state perturbation is 
the disjoint union of two smaller perturbations. As expected the 
stable states of the union are the union of the stable states, i.e. 
{x, z}, but whereas the outgoing scaling applied to the bottom of 


1 Function Hub is 

input : {S, P), where P : S x S ^ F 

// (T’, 0, 1, •, <, 3-) is an ordered-division 

semiring. 

output: a subset of S 

2 S <— {{s}|s € S'}; // For bookkeeping. 

3 for I, y G S do P({a:}, {y}) •<—P(a;,y); // For 

bookkeeping. 

4 return HubRec (S,P); 

5 end 


6 Function HubRec is 

input : (S, P), where S is a set of sets and 
P:SxS ^ F 
output: a subset of S 

7 M •<— max{P(x, y) \ {x,y) G S x S A x y}; 

8 if M = 0 then return US; // Recursion base case 

9 for x,y G S and x y do P{x, y) G- P{x, y) -G M; 

11 Outgoing scaling. 

10 A {{x,y) G S X S \ P{x,y) = 1}-, // A ±s a 

digraph. 

11 (Pi,. . . , Pj,) G-TarjanSinkSCCCSjA); // Returns 
the sink SCCs of A. 


12 


// Maximal labels of direct arcs, below, 
for 1 < i, j < fc do 

P'(UPi,UPj) max{P(2:,y) | {x,y) G Pi x Pj}; 


// Maximal labels of all relevant paths, in 
the remainder. 


13 

14 

15 

16 


17 

18 


T <— S \ (Pi U • • • U Pfe); 

Pt <— P', // Initialisation, 

for {x, y) G {S \ T) X S do Pt{x, y) ■<— 0; // Drops 
arcs not starting in T. 

for y G T do Pxiy, -) Dijkstra(S,PT,y, •, max); 
// Priy,-) is the "distance" function from 
y GT , using • and max. 
for 1 < L i ^ k and i j and 
(xi, Xj,y) G Ei X Ej X T do 
P'(UPi,UPi) ^ 

max{P'{uEi,UEj),P{xi,y) ■ Priv^Xj)); 


19 end for 

20 return HubRec ({UPi,. .., UP*}, P') 

21 end 


Algorithm 1: Hub 
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(e) Outgoing 
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(f) Transient 
deletion 


Figure 7: The algorithm run on a disconnected perturbation 


Figure [TQ (the perturbation restricted to {z, t}) would yield the 
bottom of Figure 17^ directly by division by [e®], two rounds of 
ougtoing scaling lead to this stage when processing the four-state 
perturbations. 

4. Discussion 

This section studies two special cases of our setting: first, how 
assumptions that are stronger than Assumption |5] make not only 
some proofs easier but also one result stronger; second, how far 
Young’s technique can be generalized. Then we notice that the 
termination of our algorithm defines an induction proof principle, 
which is used to show that the algorithm computes a well-known 
object when fed a strongly connected graph. Eventually, we discuss 
how to give the so-far-informal notion of time scale a formal flavor. 

4.1 Stronger assumption 

Let us consider Assumption 1391 which is stronger version of As- 
sumption|5] Assumptionl39lvields Proposition|40l which is stronger 
version of Proposition llblll (The proofs are similar but the new one 
is simpler.) 


Lemma |4^ below is a generalization of Cl, Lemma 1]. Both 
proofs use the Markov chain tree theorem, but they are significantly 
different nonetheless. Let p be a perturbation with state space S. As 
in iTll or (Hi, for all a: € S' let Tx be the set of the spanning trees 
of (the complete graph of) S x S that are directed towards x. For 
all a; e S let := maxreTx Htz.tjeT ^)- 

Lemma 42 A state x of an irreducible perturbation with state 
space S is stable iff ^ P^ for all y £ S. 

Assumption |5] and Lemma together yield Observation a 
generalization of existing results about existence of stable states, 
such as d, Theorem 4]. The underlying algorithm runs in time 
0(n®) where n is the number of states, just like Young’s. 

Observation 43 Let a perturbation p on state space S satisfy 
Assumptions^ If for all x y the map p{x, y) is either identically 
zero or strictly positive, p has stable states. 

The stable states of a perturbation are computable even without the 
positivity assumption from Observation|43 but their existence is no 
longer guaranteed by the same proof. In this way. Observation |44] 
is like the existential part of Theorem[3 but with a bad complexity. 

Observation 44 Let F be a set of perturbation maps of type I 
[0,1] for some I. Let us assume that F is closed under multiplica¬ 
tion by elements in F and by characteristic functions of decidable 
subsets of I, that Y is decidable on F x F, and that the supports of 
the functions in F are uniformly decidable. If f g or g A f for 
all f,g £ F, stability is decidable in 0{n^) for the perturbations 
p such that X 7^ p => p(x, v) £ F. 

The assumption f Ip g or g f for all f,g £ F from Observa¬ 
tion |44] corresponds to Assumption |5] Proposition 145 1 below drops 
it while preserving decidability of stability, but at the cost of an ex¬ 
ponential blow-up because the supports of the maps are no longer 
ordered by inclusion. 

Proposition 45 Let F be a set of perturbation maps of type I —>■ 
[0,1] for some I. Let us assume that F is closed under multiplica¬ 
tion by elements in F and by characteristic functions of decidable 
subsets of I, that A is decidable on F x F, and that the supports 
of the functions in F are uniformly decidable. Then stability is de¬ 
cidable for the perturbations p such that x y ^ p{x, y) £ F. 


Assumption 39 If x y and p{x,y) is non-zero, it is positive; 
and f — g or f £ o{g) or g £ o{f) for all f and g in the 
multiplicative closure of the e i—>■ Pe(x, y) with x y. 

Proposition 40 Let a perturbation p with state space S satisfy 
Assumption 1391 and let p be a stationary distribution map for p. 
Ify is a transient state, lime_).o ps{y) = 0. 

Under Assumption some states may be neither stable nor fully 
vanishing: y in Figure Island x in Figure [T^where the bottom is 
replaced with e. Assumptionl39lrules out such cases, as below. 

Corollary 41 If a perturbation p satisfies Assumption every 
state is either stable or fully vanishing. 

4.2 Generalization of Young’s technique 

Our approach to prove the existence of and compute the stable 
states of a perturbation is different from Young’s approach ca 
which uses a finite version of the Markov chain tree theorem. In 
this section we investigate how far Young’s technique can be gen¬ 
eralized. This will suggest that we were right to change approaches, 
but it will also yield a decidability result in Propositionl45l 


4.3 What does Algorithm[T]compute? 

Applying sequentially outgoing scaling, essential collapse, and 
transient deletion terminates. So it amounts to an induction proof 
principle for finite graphs with arcs labeled in an ordered-division 
semiring. Observationl46lis proved along this principle. It can also 
be proved by a very indirect argument using Lemma 1^ and Theo- 
rem[38] but the proof using induction is simple and from scratch. 

Observation 46 Let {F, 0,1, ■,<,£-) be an ordered-division semir¬ 
ing, and let P : S X S ^ F correspond to a strongly connected 
digraph, where an arc is absent iff its weight is 0. Then Hub(S', P) 
returns the roots of the maximum directed spanning trees. 

Note that finding the roots from Observation 1461 is also doable in 
0{n^) by computing the maximum spanning trees rooted at each 
vertex, by 01 which uses the notion of heap, whereas Hub uses a 
less advanced algorithm. 

Observation |4^ may be extended to non strongly connected di¬ 
graphs by considering the sink SCCs independently, but alterna¬ 
tively it is not obvious how to generalize the notion of maximal 
spanning tree into a notion that is meaningful for non-strongly con¬ 
nected graphs. Nevertheless, the vertices returned by Hub)^, P) are 
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Figure 8: Vanishing time scale 


the one in S that somehow attract the more flow/traffic according 
to P, hence the name Hub. 

One last algorithmic remark: from the proof of Proposition 
we see that Tarjan’s algorithm is an overkill to get a complexity of 
0(n®). Indeed, combining several basic shortest path-algorithms 
would have done the trick, but using Tarjan’s algorithm should 
make the computation of Hub faster by a constant factor. 

4.4 Vanishing time scales 

Under Assumption[5] computing Hub and considering the interme¬ 
diate weighted graphs shows the order in which the states are found 
to be vanishing. Under the stronger Assumptionl39l a notion of van¬ 
ishing time scale may be defined, with the flavor of non-standard 
analysis Gl . Let (T, •) be a group of functions I —t-jO, -|-oo[ such 
that / — 5 or / £ o[g) or gr £ o(/) for all / and g in T. The ele¬ 
ments of \T\ are called the time scales. Let a perturbation p on state 
space S satisfy As sumption [53 and let x £ 5 be deleted at the d-th 
recursive call of Hub(S', [p]). Let Mi,..., Mj, be the maxima (i.e. 
M) from LinejTjin Algorithm [T] at the lst,...,d-th recursive calls, 
respectively. We say that x vanishes at time scale ni<i<d M-\ 
Figure |3 suggests that a similar account of vanishing times 
scales, even just a qualitative one, would be much more difficult to 
obtain by invoking the Markov chain tree theorem as in d. The 
only stable states is f; the state 2 vanishes at time scale and 

X and y vanish at the same time scale [1] although the maximum 
spanning trees rooteed at x and y have different weights: e* and e®, 
respectively. 
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A. Tarjan Modified 

The function TarjanSinkSCC is written in Algorithm |2| It con¬ 
sists of Tarjan’s algorithm 111,111, which normally returns all the 
SCCs of a directed graph, plus a few newly added lines (as men¬ 
tioned in comments) so that it returns the sink SCCs only. It is not 
difficult to see that the newly added lines do not change the com¬ 
plexity of the algorithm, which is OdS'l -F |A|) where ISI and |A| 
are the numbers of vertices and arcs in the graph, respectively. The 
new lines only deal with the new boolean values v.sink. These 
lines are designed so that when popping an SCC with root v from 
the stack , the value v.sink is true iff the SCC is a sink, hence the 
test at Linej^ All the v.sink are initialized with true at Line|3 
and v.sink is set to false at two occasions: at Linel36lbefore a sink 
SCC with root v is output; and at Line 1241 when one successor w 
of V has already been popped from the stack (since w.index is de¬ 
fined), which means that there is one SCC below that of v. These 
are then propagated upwards at Line[T3 The conjunction reflects 
the facts that a vertex is not in a sink SCC iff one of its successors 
in the same SCC is not. 

B. Proofs and Lemmas 

Lemmal47]below relates to DefinitionjS] 

Lemma 47 1. A is a preorder and = an equivalence relation. 

2. For all f,g '. I ^]0,1], we have f g ijf ^ so g ijf 
^ 1 
f ^ g' 

3- f :i,g and f A g' implies f + f g + g' and f ■ f g ■ g'. 
4. f = g and f' = g' and f 22, f' implies g A g', 

5- / + /' = max(/, /') := X !-)• max(/(x), /'(x)). 

6- f ii f implies max(/, /') ~ /'. 

7- / I g \ j and f \i\j 2 < g |i\j implies f 22 g- 

8. Let 0 be a limit point of both J C / and 1 \ J. A state x is 
stable {fully vanishing) for a perturbation p iff it is stable (fully 
vanishing) for both p \j andp 
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1 Function Tar j anSinkSCC is 

input : {S, A), where A C S x S 
output: the sink SCC 

2 index <— 0; 

3 stack •<— 0; 

4 for V £ S do v.onstack •<— false; 

5 for V £ S do v.sink £- true; II Newly added. 

6 for w € y do 

7 I if V. index is undefined then StrongConnect (w); 

8 end for 


9 

10 

11 

12 

13 

14 


Function StrongConnect (u) is 

v.index <— index; 
v.lowlink ■£- index; 
index ■£- index + 1; 
stack.push(v); 
v.onstack ■£- true; 


15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 
27 


for (v, in) G A do 

if w.index is undefined then 
StrongConnect(w); 
v.lowlink £- 

mm{v.lowlink, w.lowlink); 

v.sink £- v.sink A w.sink; // Newly 

added. 

else 

if w.onstack = true then 
v.lowlink ■£- 
mm{v.lowlink, w.index) 

else 

I v.sink ■£- false; II Newly added. 

end if 
end if 
end for 


28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 end 


end 


if v.lowlink = v.index then 
start a new SCC; 

repeat 

w £- stack.popif); 
w.onstack <— false; 
add w to the current SCC; 

until w = v; 

if v.smfc then // Newly added. 

v.sink ■£- false; II Newly added, 

output the SCC; 

end if 
end if 


Algorithm 2: modification of Tarjan’s SCC algorithm 


Proof [Observation m Let S := {xi ,..., Xn, yi,..., ym.}, for 
alH < n — 1 let p{xi, Xi+i) := fi, let p(xn, yo) ■= fn, for all i 
letp(xi,xi) := 1 -/i, for all j < m - 1 let i/j+i) := gj, 
let p{ym, xo) := 5m, for ally let p{yj,yi) ~ 1- gj. It is easy to 
check that = Hy ffj ' ni<fe<i fi ' Y\i<k !-/»=) = 

rij 9j ■ ni<fc<i fi for sU i. Let us first assume that the functions 
in F are positive So by Lemma |42l if one Xi is stable, so is a; i. 
Likewise for the yj. But and are non comparable by 
assumption, so by Lemma|42]there are no stable state. 

Let us now prove the claim in the general case, which holds 
if n = m = 1 for the same reason as in Figure [T^ so let us 
assume that n + m > 2. Wlog let us also assume that any two 
products of n' and m' functions from F with n' + m' < n + m are 
comparable. So the functions in F are pairwise comparable; (up to 
intersection of I with a neighborhood of 0) their supports constitute 
a linearly ordered set for the inclusion; and fi ^ ni<j 9j since 
5i < 1 and ^(Ilj 9i Hi /»)• Up to renaming let us assume that 
5i has the smallest support J. Up to restriction of I to the support 
of fi, which does not change stability nor non-comparability of 
Hi fi and n ■ gj by Lemmas l47l8l and l47l7l let us assume that the 
fi are positive, and so is gj for y > 1 since Yli fi 7D ni<y 9i- 
If gi is also positive, we are back to the special case above, so let 
us assume that it is not. On the one hand, restricting p to I \ J 
shows that only yi might be stable, by Lemma 147181 on the other 
hand . gj | jA fi | j), so let us make a case disjunction: if 
Hi fi UT) Hj 9j I J then ), so yi cannot be stable by 

Lemma l42l if they are not comparable, the special case above says 
that the p \ j has no stable state, so neither has p by Lemma l47l8l □ 


Proof [Lemma|^ Let us assume that p. is stationary, so it is well- 
known that its support involves only essential states. To prove that 
the equation holds let us make a case disjunction: if x and y are 
transient states, p{x) = p{y) = 0; if a; is transient and y is 
essential, p{x) = 0 and < r^) = 0; if x and y belong 

to distinct essential classes, P^{Ty < rif) = < r^) = 0; 

if x and y belong to the same essential class E, let Ei,..., Ek 
be the essential classes. Then for all i £ {1,... , A:} let be 
the extension to S (by the zero-function outside of Ei) of the 
unique stationary distribution of p So by 

(J, Proposition 2.1] and since p is a convex combination of the 

PEi , . . . , pEy . 

Conversely, let us assume that the support of p involves only 
essential states and that the equation holds. Let E[,..., E'f., be the 
essential classes with positive p-measure. Let i < k', and for all 

X £ E i let Pi {x) ■.= ^define a distribution for p \e;xe'- 

Since pi also satisfies the equation and thatp \e;xe; is irreducible. 
Pi is the unique stationary distribution forp \e'xe'- Since p is a 
convex combination of the pi,..., pk ’, it is stationary for p. □ 


Proof [Lemma|3 


1. Let (jjv := sup{n £ N : |{A: < n : Xf^ £ ,§}| = N} 
be the supremum of the first time that has vis¬ 
ited N states in S. Clearly crjv jq P^{Ty < 

pjj") = lim]v-»oo < min(r,jf', (Tat)). On the other hand, 

P“'(r+ < min(r+,(Jiv)) 


Betz, Le Roux, Stable states of perturbed MC 


10 


2016/2/15 






















= P"(X^+ =y) 

^S\T 

+ 'Y' + = 2;)P“'(r+ < min(Tj',aiv-i)) 

Z—/ 'S\T 

zGS\{TU{x,y}) 

= p{x,y)+ p(x,z)P''(r+ < min(T+,(Tjv-i)) 

z6S\(TU{a:,!,}) 

thus by iteration we obtain 

N-l 

-E E p{x,zi)p{zi...zk)p{zk,y) 

k = l zi....,z^eS\(TU{x,y}) 

= < min(r+, o-jv)) < r+) 


ThusP“=(ra < T+) = P^^Ty < t+). 

2. Let us first assume that p is irreducible, and so is p. Let p and p 
be their respective unique, positive stationary distributions. By 
Lemmas |6] and Lemma 17111 we find 

KV) < r+) pjy) 

p{x) ¥y{T+<T+) py{T^ <Y) 

Summing this equation over y £ S proves the irreducible case. 

To prove the general claim, let Ei , Ek be the essential 
classes of p, so the essential classes of p are the non empty 
sets among Ei D S,..., Ek f) S. For i < fc let pi (pi) be 
the extension to S (S) of the unique stationary distribution of 
the irreducible p (p Is nSxB ns)- L®*- M be a station¬ 

ary distribution for p, it is well-known that p is then a convex 
combination X]i<i<fe it is straightforward to check 

that the convex combination p Y'’ . 

Pi witnesses the claim. Conversely, let p be a stationary dis¬ 
tribution of p, so it is a convex combination AA*’ 

and it is straightforward to check that the convex combina¬ 
tion p := X]i<i<fc witnesses the claim, where Li 




' Uj^i for any G S n Ep 


□ 


Proof [Lemma Let us first prove the claim for irreducible 
perturbations, and even in the following simpler case: let a; G S 
be such that for all y and all 2 7^ x we have p{x, z) = p{x, z) and 
P{z,y) = P{z,y)\ soP^(ry < T+) = P''(t;, < r+) for all z x 
since the paths leading from 2 to y without hitting x do not involve 
any step from x to another state. So 


P“'(t+ < t+) = Y P(.x,z)¥\Ty < Y) 

z^S\{x} 

- E P(.x^^W{Ty <Ti)= P'^(t-/ < r+) 

z^S\{x} 

So by Lemmas [6l 147 121 and l47l3l and since the unique p and p are 
positive. 


pjx) ^ P^(t+ < r+) ^ P^(r+ < T+) 
p(y) ~ P"(t+ < T+) “ P-(r+ < r+) 


A(a) 

Kv) 


for all y £ S', 


invoking this equation above twice shows that 

pjz) ^ ^ . AW ^ AW f ^ p c, 

p{y) p{x) p(y) p{x) p(y) p{y) 


and summing this second equation over z £ S yields , 

i.e., p{y) = p{y) for all y £ S. The irreducible case is then 


proved by induction on n ~ |{x £ S : 3y £ S,x y A 
p{x,y) z/z p[x,y)}\, which trivially holds for n = 0. For 0 < n 
let distinct x,y £ S' be such that p(x,y) 7^ p(x,y), and for all 
y,z £ S X S \{x} let p(z,y) := p(z,y), and p(x,y) ~p(x,y). 
By the simpler case p = p; by induction hypothesis /t = /i; so 
p = pby transitivity of =. 

Let us now prove the general claim by induction on the number 
of the non-zero transition maps of p that have zeros. Base case, all 
the non-zero maps are positive. Let E[ ..., E'f., be the sink SCCs of 
the graph on S with arc {x, y) if p{x, y) is non-zero. For i < k' let 
pi {pi) be the unique stationary distribution map of the irreducible 
P Ib'xb[ (A Ib'xb')-S ince p(x,i/) |b'xb:= A(*, 1 /) Ib[xb' for 
all X ^ y, the irreducible case implies pi = pi. Clearly ^ is a 
convex combination of the pi,and the convex combination p of the 
Pi with the same coefficients is a stationary distribution map for p, 
and p = pby Lemma l47l3l 

Inductive case. Let p{z, t) be a non-zero function with support 
J <Z I. Up to focusing we may assume that 0 is a limit point of 
both J and 1 \ J. By induction hypothesis on p |j= p \j and 
P |r\J— A |r\j we obtain two distribution maps pi and pi\j that 
can be combined to witness the claim. □ 

Proof [Observation 1 101 Let p{x, y) be in the essential graph. By 
the definition of 'A and finiteness of the state space, let positive 
bxy and t^y be such that p{x, 2) < fexi/ • p{x, y) for all e < t^y 
and 2 G S. So for all e < e^^y we have 1 = '^^^gPt{x,z) < 

I S'! • bxy -pe {x,y). Now let b (eo) be the maximum (minimum) of the 
bxy (Cxy) for (x, y) in the essential graph. Thus 0 < (6- < 

^£( 7 ) for all e < eo- □ 

Proof [Lemma [T2I For all y £ T let Xy be an essential state 
reachable from y in the essential graph. So c • p{y) < p{xy) 
for all y G r by Lemma[^ Therefore 1 < c~^ Tly^T Pi^y) + 
'Yxx<zs\t pi^)’ ®rid the claim is proved by further approximation. 

□ 

Lemmal48l below is a technical tool proved by a standard argument 
in Markov chain theory. 


Lemma 48 Let a perturbation with state space S satisfy Assump- 
tion\n\ and let x be in some essential class E. Then for a// n G N 




Proof Let r* := For every y £ E let 'yy £ 

TE{y, x) be in the essential graph, c < pijy) by Assumption [TT] 
so maxj,gB Pj (r* > |S|) < 1 — c < 1, so for all fe G N 

P"(t* > k\S\) < (maxP^W > \SY 

yeE 


< (1 — c)*' by the strong Markov property, so 

“(r* > n) < P'^(t* > l^l • Lt^J) ^ (1 “ for all n G N. 

|7| 


□ 


Proof [Lemma [TS) Up to focusing let p satisfy Assumption 1111 
The second statement boils down to Lemma I47I3I For the first 
statement, for every z £ E let jz £ Te{x, 2) be a path in the 
essential graph, so c < pi'Jz) by Assumption [TT] Let r* := 

Ys\E)u{xy 

c-p{z,y) < c-¥^{Xr- =y) <p{'yz) = y) 

=y)=p{uE,y) 

which implies c • maxzeE p{z,y) < p{UE,y) i.e. half of the 
statement p(UiJ, y) = maxz^sp{z,y). Now, for every positive 
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natural N let Ajv := {x} x {E\ {x})'^ ^ x {y}. 

= y,T* = N) ^ p(j) 

76Ajv 

<ma,xp{z,y) V p{y) 

z^E 

■yteA^ 

= msLxp{z,y)¥^{T* > N — 1) 

z^E 

Let g := (1 - c)'®' < 1, so P"^(r* > iV) < (1 - c)"^ ■ by 
Lemma l^ and 

OO 

P"(X,. =y)=Y =y^^* = 

N=1 

OO 

< maxp{z, y) V (1 - cY ■ 

z^E ' ^ 

N=0 

= (1 - c)”^(l - g)"^ • ma,xp{z, y) 

z£E 

□ 


Proof [Proposition 1161 Up to focusing let p satisfy Assump¬ 
tion [TT] 


1. Let y be in the set of the transient states T, so there exists x 
and 7 G VT{y,x) such that 7 is also in the essential graph. By 
Assumption! 11 Ithis implies 


0 < liminfpe( 7 ) < limin; 


< 4). 


On the other hand, let E be the essential class of x. Lemma [TS] 
implies that 


<T + ) < 1-P"(X^+ =X) 

’^S\EU{x} 

= P“'(A + = a) = maxp(f, 2) 

^ ^ ^s\Eu{x} ^ te£ ^ ^ 

z^E z^E 


stays inside. 

< Tub) = V'^ixy < Tz:,Ty < Tb_i) 

by definition of the essential collapse, and te,i 
< f^ixy < Xx), showing half of the first conjunct. 

For the other half, let X satisfy Assumptionll llup to focusing. For 
all 2 G i? \ {a;} there is a simple path in the essential graph from 
a; to 2 , so c < P^(tz < ts\bu{i})> by the strong Markov 
property 

< Xz:, Xy < Xe,i) > P"" (Tz < Xs\EU{z:}) ' {Xy < Xz:, Xy < Xe,i) 

> C ■F'^iXy < Xz;,Xy < Xe,i) (1) 

For all 2 G E \ {a;} there is also a simple path in the essential 
graph from 2 to x, so F^{xs\e < x^) < 1 — c. Also note that 
F^{x% i < Xx) < F^{xs\e < Xx), and let us show by induction 
on n that F^{xe^„ < Xx) < (1 — c)", which holds for n = 0. 

P'^jr^ „+i < Xx) < P'^j-rs „ < Xx) ■ maxP^(rB 1 < Xx) (2) 

z^E 

by the strong Markov property. 

< F^{xE,n < Xx) • (1 — c) by the remark above. 

< (1 — by induction hypothesis. (3) 

Let us now conclude about the second half of the first conjunct. 

OO 

F'^iXy < Xx) = y^P'^(Ta < Xx,XE,n < Xy < XE,n+l) 
n=0 

by a case disjunction. 

OO 

< < Xx,XE,n < Xx,Xy < XE,n+l) 
n=0 

since the new conditions are weaker. 

OO 

< V'P'^(rB „ < Ta:) maxP"’(T;, < Xx,Xy < Xe,i) 

Z^E 

n = 0 

by the strong Markov property. 


Since E is an essential class, for all f G FI and z ^ E the func¬ 
tion p(t, 2 ) is not 7 j-maximal. So lim mfe_»o Pe(f, 2 ) = 0 by 
definition of the essential graph, and lim inf e_»o 'YIiz^e maxtgsPejf, z) = 
0 by Assumption|5] Lemma l47l6i and finiteness. By combining 
this with the two inequalities above one obtains 


< c ^F^{xy < Xx,Xy < xe,i) ■ y^(l - c)" 

n=0 

by inequalities [T] and [ 3 ] 

< c~^F^^{xy < xue) 


lim inf 

e-»0 


FUx+<x+) 

Pe (Xx < Xy ) 


= 0 


and noting the following by Lemma|^allows us to conclude. 


y^iv) < 


p^y) FUx+<x+) 
Me (x) P| (t^ < t/) 


2. By Assumption [TT] (twice) and Lemmaj^ 


□ 


Proof ILemma fTTI Let us first prove the first conjunct. On the one 

hand F'‘^{xx < Xy) < F^{xe < Xy) = P“(Ti < Xy), and on the 
other hand P^jrx < Xy) > P^jrs < Xy) ■ min^gB P^(ra: < Xy) 
by the strong Markov property. Since lime->o F^{xx < Xy ) = 1 for 
all 2 in the essential class E, F^{xx < Xy) and F'^{xx < Xy) are 
even asymptotically equivalent. 

Let us now prove the second conjunct. Let Xe 0 '■= 0 and 
XE,n+i inf{f > XE^n I G E A 3s G]'rB „,f[, Xs (fz E} 
for all n G N. Informally, when starting in E, x% ,^ is the first time 
that the chain is in E after n stays outside that are separated by 


□ 

Proof [Proposition 1181 Up to focusing let p satisfy Assump¬ 
tion [TT] 

1. Let us first assume that p is irreducible, and so is p by Observa- 
tion[T4] So both p and p have unique, positive stationary distri¬ 
bution maps. Let us prove that m(F) = p{x) where x := VJE 
and ft{y) = p{y) for all y G S' \ i?. For y £ S \ E, Lemma]^ 
and LemmafTT] imply the following. 

Ky) ^ F^jxy < x^) ^ F^{x+ < x^) ^ mM (4) 
p{x) Fy{x+<x+) Fy{x±<x+) y{x) 

Summing the above equation over y £ S \ E and adding 
tM ^ ^ where p := 

X] 2 gB\{a:} f(^)' So by Definition]^ let a and b be positive real 
numbers such that a ■ jl{x)~^ < (1 ~ P-)y{x)~^ < b ■ fl{x)~^ 
on a neighborhood of 0, which yields a ■ p.{x) < p{x) < 
b-p{x) + fj,. Since fl < b' -uix) for some b' by Proposition [T6l2l 
fi{x) = p{x). Now by Lemmas 14712! and l47l3] let US rewrite 
fi{x) with fi{x) in Equation |4] which shows the claim for 
irreducible perturbations. 
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Let us now prove the general claim by induction on the number 
of the non-zero transition maps of p that have zeros. Base case, 
all the non-zero maps are positive. Let E[ ..., E'y be the sink 
SCCs of the graph on S with arc {x,y) if p{x,y) is non¬ 
zero. The essential graph is included in this digraph, and the 
essential class E is included in E'j for some j £ {1,. .., k'}. 
Let := {iJ} U Ej \ E and for all i ^ j let S,' := E'i. 

For all i £ {1,..., fc'} let (Pe') be the extension to S 

({i?} U S' \ E), by the zero-function outside of S' (El), of the 
unique stationary distribution of p \e'xe' (P \e'xe'^’ 
the irreducible case above let p,p,i (p e'. ) be the corresponding 
unique distribution of p |{_ b } ub'\b after (p \e'.xe'. before) 
the essential collapse. Since p (p) is a convex combination 
Y.i<i<k' <^iPE[ aiP-sO’ it is easy to check that 

p '.= ^^Pe^ ^2i<i<k' ^^Pe'^( is ^ witness 

for the base case. 

Inductive case. Let p{z, t) be a non-zero function with support 
J £ I. Up to focusing we may assume that 0 is a limit point of 
both J and 7 \ J. By induction hypothesis on p | p | j and 
p |/\j= p |/\j we obtain two distribution maps pi and pj\j 
(pi and pi\j) that can be combined to witness the claim. 

2. By Propositions ! 1 61 1 l and l 1 81 1 l in both cases, and also by Propo- 
sition llhl^ if y £ E. 

□ 


where the numerator may be decomposed as 
p{x, y) + E.sa\{.} ^)iP"(^-(s\A)u{.} = !/)• By the induc¬ 
tion hypothesis for the set A\{a:} let us rewrite ~ 

p) for all 2 G A \ {a:}, and obtain the equation below that may be 
re-indexed to yield the claim. 


F^(X. 


'^S\A 


y) 


_ p{x,y) 

1 -P^(X + 

’■(S\A)U{x} 


x) 


+ E 

z£A\{x},p(x,z)>0 


E 

T'6rA\{x}(^.v).P(7)>0 


_ p{x,z) ■ n 

1 _ px (X + 

^(S\A)U{x} 


x) 


where If 


hl-i 

n 


i=l 


_ pinni+i) _ 

l_P7.(X^+ =7i) 

(S\A)U{x,7i .7j} 


□ 


Proof lLemma l22l Up to focusing let p satisfy Assumption M 
Let T* ■.= and consider 

= y) =p{x,y) + ^p(a;, 2 )P"'(Xx* = y). 

zeT 

For all 2 £ r there are 2 ' £ S \ T and 7 ^ £ Tt{z, 2 ') in the 
essential graph, so for all 7f C 5 we have P^ {X + = z) < 

'^(S\T)UK _ 

P^{X + = z) < 1 — pH) < 1 — c. So by Lemmald^ 

^(S\T)U{x} 


Proof [Proposition 1211 Up to focusing let p satisfy Assump- 
tion[TT] By induction on the number of the non-zero transition maps 
of p that have zeros. Base case, all the non-zero maps are positive. 
Let E[ ..., E'f,, be the sink SCCs of the graph on S with arc (a;, y) 
if p{x, y) is non-zero. Note that this digraph includes the essential 
graph, and that 5{p \e'.xe0 = S{p) U ' ns \ Txi5 ' ns \ T - Moreover, 
by Lemmas I7l2l the stable states of each p \e'xe' are also stable 
for S{p \e'xe'.)^ and the converse holds by Lemmas |7|2| and [T^ 
Therefore a state is stable forp iff it is stable for some p \e'xe' iff 
it is stable for some S{p) |B'ns\TxB<ns\T iff it is stable for 5{p). 

Inductive case. Let p( 2 , t) be a non-zero function with support 
J £ I. Up to focusing we may assume that 0 is a limit point of 
both J and 7 \ J. By induction hypothesis p | j and 5(p) \j have 
the same stable states, and likewise forp j and 5{p) |/\j, which 
shows the claim. □ 

Lemma |4^ below is a generalization to the reducible case of 
Proposition 2.8 from jlll . 


Lemma 49 Let A be a finite subset of the state space S of a 
Markov chain. Then for all x £ A and y £ S \ A 


= y) = 

E76r.4(x,!/),p(7)>0 




i-p7i(x + 


Proof We proceed by induction on | A|. The claim trivially holds 
for I A| = 0; so now let x £ A. The strong Markov property gives 


¥^{X 


'^S\A 




+ 

(S\A)U{x} 


+ V^(X + 

'^(S\^)U{x} 


= y) 


y) 


If p( 7 ) = 0 for all 7 £ rA(a;, y), the claim boils down to 0 = 0, 
so let us assume that there exists 7 £ TA{x,y) with p( 7 ) = 0 , 
so ¥^(X + = a;) < 1 , and the above equation may be 

''■(S\A)U{x} 

rearranged into 


= y) = 


P"(X^+ =y) 

_^(S\A)U{x}_ 

1 _px(X + =x) 

’■(S\A)U{x} 


¥^{Xr»=y)< P( 7 )'C '"^'<0 p(7). 

■yeriiz.y) ~ferT(z,y) 

Since P'=(Xx* = y) > E^gr^ u,^) K t). thus P^(Ax» = p) = 
E 7 grr(z y) Pil)’ ™ by Lemma R7l5l we can replace the sum with 
the maximum. □ 


Proof [Observation [24l Let e £ 7. If g{e) fz Q then ((/ 

9 ) ■ 5)(e) = • 9(e) = /(e); if 9(e) = 0 then /(e) = 0 = 

if g){e) ■ g{e). □ 


Proof [ProDosition l26l Let m be as in Definition|^and let J C 7 
be its support. 


1. a{p){x,y) | 7 \j= [S'! ^ for all®,p £ 5, so cr(p)e is stochastic 
foralle£ 7\J.Lete£ J. OntheonehandEj,gs er(p)E(®,p) = 


Pe (x,x) + m—l 


Pe(z:,y) 


'P EygS\{x} 

EugS\{x} ^ip)<^ix, v) = E;,gs\{x} |S|-max{p,(t,z) | z.tgSAz^i} 


on the other hand 

Pejx.y) 


< 1, so cr(p)e is stochastic. If 0 is not a limit point of 


J, it is clear that (j(p) (x, p) = 1 for all ®, p £ S, and cr(p) sat¬ 
isfies Assumption[3 If 0 is a limit point of J, then p{x, y) | j A 
P{x',y') IJ implies cr(p)(®,p) \jfl a{p){x',y') \j (since mis 
non-zero on J), so o'(p) satisfies Assumptionl^by Lemma l47l7l 
(since p |j and then (j(p) |j do). 


< 


2. Let us first prove the claim if J = 7. The equation below shows 
that p • p = p iff p • ct(p) = p, for all distribution maps p on 
S = {xi,..., Xn}, so p and (j(p) have the same stable states. 


(9 • fx{p))j 


p{xj^Xj) + m — 1 
m 


• 97 + E 


p(Xi,Xj)pi 

m 


{m-l)pj +J2iP{xi,Xj)pi 
m 


Let us now prove the claim if J C 7. The case where 0 is 
not a limit point of 7 \ J amounts, up to focusing, to the case 
J — I, so let us assume that 0 is a limit point of 7 \ J. For 
all e £ 7 \ J we have pe{x,y) = 0 for all states x y, 
so all distributions are stationary for p^. Moreover, the uniform 
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distribution is stationary for (T(p)e, so, all states are stable for 
p |j\j and cr(p) \i\j- Therefore, if 0 is not a limit point of J, 
we are done, and if it is, we are also done by Lemma l47l8l 

3. Let distinct x,y £ 5 be such that p{z,t) ^ p{x,y) for all 
distinct z,t £ S. So p{x, y) = msix{p{z, t) \ z,t £ S A z ^ 
fjby Lemma |47|6l soa{p){x,y) p{x,y)^^s\\S\-p{x,y) = 
= 1 , so {x, y) is in the essential graph of cr(p). 

□ 



Lemma 50 Let a perturbation p with state space S satisfy As- 
sumption\^ let Ei,, Ek be its essential classes, and for all i 
let Xi £ Ei. The state x is stable for p iff x belongs to some Ei 
such that yjEi is stable for 5 o k{. .. K.{ti{a{p),xi), X 2 ) ■ • ■, Xk). 

Proof lTheorem l27l By applying Lemma |5^ recursively. If p is 
the identity matrix then all states are stable. Otherwise the essential 
graph of cr(p) is non-empty, so either one essential class is not 
a singleton, or one state is transient. If there is a non-singleton 
essential class, the corresponding essential collapse decreases the 
number of states; if one state is transient, the transient deletion 
decreases the number of states. Since these transformations do not 
increase the number of states, Soii{... K{K{a{p),xi),X 2 ) . ■., Xk) 
has fewer states than p, whence termination of the recursion on an 
identity perturbation whose non-empty state space corresponds to 
the stable states of p. □ 



3 . The essential classes of p are singletons since S{p) is well- 
defined. Let {*} and {y} be distinct essential classes of p, 
and of [p] by Lemma [35111 Let M ~ max[x]{[p]( 7 ) : 
7 G r T(3:,p )}, so M = [max{p( 7 ) : 7 G rT(a;,p)}] by 
Lemma |47 161 Note thatp(a;,a;) = 1 since {a;} is an essential 
class of p, so 5 I]zgs\T■ 7 ^ ^Tix,z)} = 1 too. 
So 


Proof [Observation!^ If / < 0 then / = (/ -G 0) ■ 0 = 0. Also, 
(/Al) = (/Al)-1 = /. □ 


Proof [Lemma 


1. By Lemma I47I7I with J the support of g and I \ J, then hy 
Lemmas 147 121 and I47I3I 

2. By Observations I29I2I and \2^ and since [e !->• 1] is the [;^]- 
maximum of [G U {e 1 —0}]. 

□ 


Proof [Lemma [35l 

1. Clear hy comparing Definitions [^and r33l II 

2. Let [p\{z,t) = max{[p]( 2 ', f') : {z',t') £ S x S A z' t'}. 

Ifxf^y then 

[a]{[p]){x,y) = [p](a;,p)[-G][p](a, 7) hy definition of [cr], 

= [p{x, y)] [-G] [p{z, t)] by definition of [p], 

= [p{3:, y) -i-|s| p{z, t)] by Lemma[3l 
= [p{3:, y) -l-|s| max{p(z, f) \ z,t£ S Azf^t}] 
by Lemmal^and Lemma I47I6I 
= [g(p)](x, y) by definition of a. 


[x](b])({®}, {j/}) = M by definition of [x], 

= M[-g][ 1] by Observation 

max{p( 7 ) : y £ Tt{x, z)}] by a remark above, 

zes\T 

= [max{p( 7 ) : 7 G rT( 2 ;,p)}lH 

[ max{p( 7 ) : 7 G rT(a;, a)}] by a remark above, 
zes\T 

= [max{p( 7 ) : 7 G rT(a;, y)}7-|s| 

max{p( 7 ) : 7 G Vt{x,z)Y\ by Lemma 

zes\T 

= [(5(p)](a:, y) by definition of 5. 
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[ 1 ] 



a[\ x,y £ S \ El, which then also holds for paths 7 G 
TriEi, Ej). Let us now show that [x]{P){UEi,UEj) = 

[x] ° Ei){UEi,UEj). On the one hand for all paths 

xy € TriEijEj) we hawe P{xy) < [k](P, i5i)((Ui5i)7) 
since [k\{P, Ei)(uEi,y) := max<{P(a;, y) : x £ Pi}, and 
on the other hand for every 7 £ T* x Ej there exists a; £ Pi 
such that [k](P, Pi)((UPi) 7 ) = P{xy). So 

[x](P)(UPi, UPj) = max{P(7) : 7 £ rT(Pi, Pj)} by definition, 

= max{[K](P, Pi)(7) : 7 £ rT(UPi, P^)} 

by the remark above, 


= [x] o [k](P, Pi)(UPi, UPj) by definition. 


5 


[x] The equality [x](P)(UPi, UPi) = [x]°M(-P, Pi)(UPi, UPi) 

can be proved likewise. 


12-3e2- 


6 . 



Also [K\([p\,Ei){\jEi,y) = ma,x[y.]{\p]{x,y)\x £ Pi} = 
[max| p(a;, y ) \ x £ Pi}] = [k,{p, Xi){UEi,y)] by definition, 
Lemma l 47 l 6 l and LemmafTfi] Likewise [rt]([j3], Ei){y, UPi) = 
max[;^]{[p](i/, x) |a; £ Pi} = [max{p(y, x) j x £ Pi}] = 
[K{p,Xi){y,UEi)]. 


Let us first prove J5 ok(. .. k{k{p, xi), X 2 )..., Xi;)](UPi, UPj) = 
[x]([p])(UPi, UPj) for all i 7 ^ y by induction on the num¬ 
ber k' of non-singleton essential classes. Since collapsing a 
singleton class has no effect, the claim holds for k' = 0 by 
Lemma I35I3I so let us assume that it holds for some arbi¬ 
trary k' and that p has k' + 1 non-singleton essential classes. 

One may assume up to commuting and renaming that Pi is 

not a singleton. Since xi), UPi) = xi), also 5 o 

k{. .. k{k{p, xi), X 2 )..., Xfc) = (5 ok(. .. k{k{k{p, xi), UPi), X 2 ). .., Xfc). 

So J(5 o k(. .. k(k(p, xi),X 2 ).. ., Xfc)](UPi, UPj) = 

[x]([«:(p, xi)])(uPi, UPj) for all i 7 ^ j by induction hypoth¬ 
esis. Moreove^x](WP,®i)]) = [x](M(b],-Ei)) = ]x](b]) 

by Lemmas r35l4l and [35l5l Therefore Jx] ([cr(p)]) (UPi, UPj) = 

J5ok(. .. k(k^(p),xi),X 2 ) ... ,Xfc)](UPi, yjEj) foralH ^ j 
by Lemma I35l2l and Observationl34l 

□ 


[ 1 ] 



5 . Let P := Jp] and <:= [A], and let us prove the claim abstractly. 
First note that Jx] o [k](P, Pi) and [x](P) have the same state 
space {uPi,..., UPfc}. For i,j 7^ 1 the definition of [x] 
gives lx] o [k](P, Pi)(uP7,UP7) = max<{lAc](P, Pi)(7) : 
7 £ rT(Pi,Pj)}, where T ~ S \ UiPi. It is equal to 
[x](^’)(UPj,UPj) since [k](P, Pi)(x,p) = P{x,y) for 


Proof [Proposition 1371 Line |2] from Algorithm [T] is performed 
once and takes n steps; Line [3 takes one step and is performed 
times. Let us now focus on the recursive function HubRec. If all 
the arcs of the input are labelled with 0, the algorithm terminates; if 
not, p{s,t) = 1 at least for some distinct s,t G S after the outgoing 
scaling at Line[^ so either the strongly connected component of s 
is not a sink, or s is in the same strongly connected component 
as t, which implies in both cases that there are fewer U5i than 
vertices in S, and subsequently that HubRec is recursively called at 
most n times for an input with n vertices. Lines in H [ml [HI [H 
M and[T5]take at most 0{n^) steps at each call, thus contributing 
0{n^) altogether. Tarjan’s algorithm and its modification both run 
in 0(]A] -F 15]) which is bounded by O(n^), and moreover the 
arcs from different recursive steps are also different, so the overall 
contribution of LinefTTIis 0{n^). 

Let us now deal with the more complex Lines [T^ and [T^ Let r 
be the number of recursive calls that are made to HubRec, and at 
the y-th call let T) denote the vertices otherwise named T. Since 
the (j -F l)-th recursive call does not involve vertices in Tj, we 
obtain \'Pj\ — The loop at Line[T^is taken at most \Tj \ 

times during the j-th call, which yields an overall contribution of 
0(n®). Likewise, since a basic shortest-path algorithms terminates 
within O(n^) steps and since it is called \Tj \ times during the j-lh 
recursive call, Linellbis overall contribution is 0(n®). □ 

Proof [Proposition 1401 Up to focusing let p satisfy Assump¬ 
tion m Let y be in the set of the transient states T, so there exist 
X and 7 £ rT(p, x) in the essential graph. By AssumptionfTT] 
this implies 

0 < liminfpe( 7 ) < lim inf (x)} < xj"). 

€—FO €—FO ^ 
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On the other hand, let E be the essential class of x. Lemma [B] 
implies that 


'(r+ <r+) < 1-P"(X^+ =x) 

^S\Eu{x} 


= P (X + = z) = maxp(t,z) 

^ ^ B\eu{x} ^ ^ teE ^ ^ 

z^E z^E 


Since E is an essential class, ^^^t£E Peit, z) 0, thus 

by Lemma 1^ 


p4y) < 


p4v) 

p4x) 


< r+) 

Pf (r) 


□ 


Proof ICorollarv 1411 In the procedure underlying Theorem 1271 
only the states that are transient at some point during the run are 
deleted. By Proposition |40] these are exactly the fully vanishing 
states. □ 

Proof lLemmal42l For all x G Sletqf ~ J^tgTx n(z 

and let q := (q^)y^s,eei- By irreducibility qj > 0 for all 2 G S 

and e G 1, so let uf := „ —w for all 2 G S and e G /. Let us 

Eyes « 

assume that j3'^ /3^ for all i/ G S', so by finiteness of S there exist 

positive c and eo such that PY < c ■ Pf for all y G S and e < eo- 
For all 2 / G S and e < eo we have 


gv 

X -v. nx -v. He 

qe > Pe > - > 



TeTy (z,t)€T 


ql 

C-\Ty\ 


Note that \Ty \ < since a spanning tree of a graph is a subset 
of its arcs, so 

X Qe ^ 1 

- c-E.esIB.I - c.|S|.2l^P 

which ensures that liminfe_>o p-c > 0. By the Markov chain tree 
theorem p ■ p = p, so x is a stable state. 

Conversely, let us assume that -^{P^ for some q G S, so 

for all c, e > 0 there exists a positive q < e such that c- Py < P^- 
Let c, e > 0 and let a positive q < e be such that ■ P^ < pi^, 

so c ■ Py < Py. Since p < 1, it shows that lim infs-jo p^ = 0. □ 

Proof [Observation 1431 let G be the graph with arc {x, y) if 
p{x, y) > 0. Let E'l,..., Ey be the sink (aka bottom) strongly 
connected components of G, so a state is stable for p iff it is 
stable for one of the p \e'.xe'- Since the p \e'.xe' irreducible 
perturbations, Lemmal4^can be applied, and by AssumptionOthe 
weights of the spanning trees are totally preordered, so there are 
stable states. □ 


Proof 10bservation l44l For all x,y G S let Ey be the support of 
p{x,y) : / —^ [0, 1]. By Assumptionthe Ey are totally ordered 
by inclusion. Among these sets let 0 C Ji C ■ • • C /; C / be the 
non-trivial subsets of I. Up to focusing on a smaller neighborhood 
of 0 inside 1 , let us assume that 0 is a limit point of Ji, all 
the li+i \ li, and / \ Ii. By Lemma I47I8I a state is stable for p 
iff it is stable for p |jj, all the p and p |/\/,. These 

restrictions all satisfy the positivity assumption of Observation 143 1 
whose underlying algorithm computes the stable states in 0{rP'). 
Since there are at most rP restrictions, stability is decidable in 
0(n®). □ 

Proof [Proposition 145 1 By induction on n := \{p{x,y) \ x 7 ^ 
y A p{x,y) / 0 A -■(0 < p{x,y))}\. If n = 0, let G be the 
graph with arc (x, y) if p{x, y) > 0. Let E[,... , E'y be the sink 
SCCs of G, so a state is stable for p iff it is stable for one of 


the p Ib'xb'- By decidability of A and since the p |b'xb' are 
irreducible perturbations. Lemma |42] allows us to compute their 
stable states. 

If n > 0 let p{x, y) be a non-zero function with zeros, and let 
J be its support. If 0 is not a limit point of J (/\J), the stable 
states of p are the stable states of Pi\j (pj), which are computable 
by induction hypothesis. If 0 is a limit point of both J and 1\J, 
by Lemma R7l8l the stable states wrt p are the states that are stable 
wrt both Pj and pj\j, and we can use the induction hypothesis for 
both. □ 

Proof [Observation 1461 By induction. More specifically, let us 
prove that these roots are preserved and reflected by outgoing 
scaling, essential collapse, and transient deletion. 

• Since the outgoing scaling divides all the coefficients by the 

same scale f G E, the weights of the spanning trees are all 
divided hy and the order between them is preserved. 

• Let be a (sink) SCC of the essential graph of P, and let 
x,y G S. It is easy to see that a spanning tree rooted at x can be 
modified (only within E) into a spanning tree rooted at y that 
has the same weight. Since the arcs in E do not contribute to 
the weight, the essential collapse is safe. 

• Let the sink SCCs of P be singletons, and let {y} not be one of 
those, so there exists a path from 1 / to a sink SCC {*}. Let T be 
a spanning tree rooted at y. Following T, let x' be the successor 
of X, so the weight of [x, x') is less than 1. Let us modify T 
into T' by letting y lead to the new root 2 ; by a path of weight 
1. The weight of T' is greater than that of T by at least the 
weight of {x,x'). This shows that only essential vertices may 
be the roots of spanning trees of maximum weights. Moreover, 
let r be a spanning tree of maximum weight, and let x and 
y be essential vertices such that following T from x leads to 
y without visiting any other essential vertex. Then this path 
between x and y must have maximal weight among all paths 
from xtoy that avoid other essential vertices. So the weight of 
maximal spanning trees after transient deletion correspond to 
the weight before deletion. 

□ 
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